The present invention relates to the field of image motion. In particular, the invention relates to a method and apparatus for segmenting moving objects and determining their motion from video image data The invention also relates to a computer program product including a computer readable medium having recorded thereon a computer program for segmenting moving objects and determining their motion from video image data.
Image motion plays an important role in computer vision and scene understanding. Image motion analysis has been applied to many fields over the last few decades, including object tracking, autonomous navigation, surveillance and virtual reality. More recently, motion information has played an important role in video indexing, contributing to video segmentation and shot classification.
The U.S. Pat. No. 5,748,761 by Chang et al discloses a method of segmenting a moving object and estimating its motion from video image data. Chang et al intially compares a current frame with a previous frame of video image data and generates a motion vector field. Chang et al then segments the motion vector field into moving objects and describes each motion of the moving objects with a set of motion parameters. This segmentation is achieved by utilising a region seeding and region growing procedure. The region seeding procedure consists of sliding a window block on the motion vector field, and computing the degree of homogeneity for each of the window blocks and selecting the most homogeneous window block as a seed block. The region growing procedure is then iteratively repeated until the difference between the current revised region and the previous revised region is less than a predetermined threshold value. Presumably, the threshold value is arbitrarily selected by a user. However, this method suffers from the disadvantage that the choice of threshold values is critical for successful segmentation of the moving objects. Specifically, a particular threshold value may work with one video image but not necessarily with others. For example, this method often fails to split regions that must be separated or fails to merge regions that need not be separated.
It is an object of the present invention to ameliorate one or more disadvantages of the background art.
According to one aspect of the invention there is provided a method of segmenting moving objects and determining their motion from video image data, wherein said method comprises the steps of: distributing seeds in areas of a current frame of video data as a function of motion of pixels of the current frame as compared to a previous frame, wherein fewer seeds are allocated to those areas of the current frame having homogeneous motion; and growing regions from said- seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion that is most similar to a motion of a region it borders is appended to that region and the motion of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to another aspect of the invention there is provided a method of segmenting moving objects and determining their motion from video image data, wherein said method comprises the steps of: comparing a current frame with a preceding frame of the video image data to compute a motion parameter field having a plurality of sets of motion parameters, wherein each pixel of the current frame has a corresponding said set of motion parameters, distributing seeds in areas of the motion parameter field as a function of the motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the motion parameter field having homogeneous motion parameter sets; and growing regions from said seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion value that is most similar to a set of motion parameters of a region it borders is appended to that region and the set of motion parameters of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to still another aspect of the invention there is provided a method of segmenting moving objects and determining their motion from video image data, wherein said method comprises the steps of: (a) comparing a current frame with a preceding frame of the video image data to compute a motion vector field having a plurality of motion vectors, wherein each pixel of the current frame bas a corresponding motion vector; (b) sliding window blocks over the motion vector field and generating a motion parameter field, wherein each pixel of the current frame has a corresponding set of motion parameters and the sets of motion parameters are based on a motion model of said motion vectors; (c) allocating pixels as seeds in areas of the current image as a function of the corresponding motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the current image having homogeneous motion parameter sets and wherein said seeds form growing regions; (d) generating a list of pixels and corresponding motion vectors that border the growing regions; (e) scanning a number of said pixels of the list; (f) determining, for each said scanned pixel, a value indicative of the similarity of the corresponding motion vector of said scanned pixel and the corresponding set of motion parameters of a growing region that said scanned pixel borders; (g) selecting a pixel that has a minimum said value; (h) appending said selected pixel to said growing region it borders; (i) updating the set of motion parameters of the appended region; (j) repeating the sub-steps (d) to (i) until there are no more pixels that border the growing regions; and (k) merging neighbouring grown regions to produce a merged region if their similarity is less than a predetermined threshold, wherein said merged regions represent said moving objects and their corresponding sets of motion parameters represent their motion.
According to still another aspect of the invention there is provided apparatus for segmenting moving objects and determining their motion from video image data, wherein said apparatus comprises: means for distributing seeds in areas of a current frame of video data as a function of motion of pixels of the current frame as compared to a previous frame, wherein fewer seeds are allocated to those areas of the current frame having homogeneous motion; and means for growing regions from said seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion that is most similar to a motion of a region it borders is appended to that region and the motion of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to still another aspect of the invention there is provided apparatus for segmenting moving objects and determining their motion from video image data, wherein said apparatus comprises: means for comparing a current frame with a preceding frame of the video image data to compute a motion parameter field having a plurality of sets of motion parameters, wherein each pixel of the current frame has a corresponding said set of motion parameters; means for distributing seeds in areas of the motion parameter field as a function of the motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the motion parameter field having homogeneous motion parameter sets; and means for growing regions from said seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion value that is most similar to a set of motion parameters of a region it borders is appended to that region and the set of motion parameters of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to still another aspect of the invention there is provided apparatus for segmenting moving objects and determining their motion from video image data, wherein said apparatus comprises: means for comparing a current frame with a preceding frame of the video image data to compute a motion vector field having a plurality of motion vectors, wherein each pixel of the current frame has a corresponding motion vector; mean for sliding window blocks over the motion vector field and generating a motion parameter is field, wherein each pixel of the current frame has a corresponding set of motion parameters and the sets of motion parameters are based on a motion model of said motion vectors; means for allocating pixels as seeds in areas of the current image as a function of the corresponding motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the current image having homogeneous motion parameter sets and wherein said seeds form growing regions; means for generating a list of pixels and corresponding motion vectors that border the growing regions; means for scanning a number of said pixels of the list; means for determining, for each said scanned pixel, a value indicative of the similarity of the corresponding motion vector of said scanned pixel and the corresponding set of motion parameters of a growing region that said scanned pixel borders; means for selecting a pixel that has a minimum said value; means for appending said selected pixel to said growing region it borders; means for updating the set of motion parameters of the appended region; means for repeating the operations of the generating means, scanning means, determining means, selecting means, appending means, and updating means until there are no more pixels that border the growing regions; and means for merging neighbouring grown regions to produce a merged region if their similarity is less than a predetermined threshold, wherein said merged regions represent said moving objects and their corresponding sets of motion parameters represent their motion.
According to still another aspect of the invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for segmenting moving objects and determining their motion from video image data, wherein said computer program product comprises: means for s distributing seeds in areas of a current frame of video data as a function of motion of pixels of the current frame as compared to a previous frame, wherein fewer seeds are allocated to those areas of the current frame having homogeneous motion; and means for growing regions from said seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion that is most similar to a motion of a region it borders is appended to that region and the motion of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to still another aspect of the invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for segmenting moving objects and determining their motion from video image data, wherein said computer program product comprises: means for comparing a current frame with a preceding frame of the video image data to compute a motion parameter field having a plurality of sets of motion parameters, wherein each pixel of the current frame has a corresponding said set of motion parameters; means for distributing seeds in areas of the motion parameter field as a function of the motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the motion parameter field having homogeneous motion parameter sets; and means for growing regions from said seeds so as to segment the current frame into a number of said objects, wherein a number of pixels that border said growing regions are considered and that pixel of said number having a motion value that is most similar to a set of motion parameters of a region it borders is appended to that region and the set of motion parameters of the appended region is updated and said growing step is repeated until no pixels bordering the growing regions are available.
According to still another aspect of the invention there is provided a computer program product including a computer readable medium having recorded thereon a computer program for segmenting moving objects and determining their motion from video image data, wherein said computer program product comprises: means for comparing a current frame with a preceding frame of the video image data to compute a motion vector field having a plurality of motion vectors, wherein each pixel of the current frame has a corresponding motion vector; mean for sliding window blocks over the motion vector field and generating a motion parameter field, wherein each pixel of the current frame has a corresponding set of motion parameters and the sets of motion parameters are based on a motion model of said motion vectors; means for allocating pixels as seeds in areas of the current image as a function of the corresponding motion parameter sets within those areas, wherein fewer seeds are allocated to those areas of the current image having homogeneous motion parameter sets and wherein said seeds form growing regions; means for generating a list of pixels and corresponding motion vectors that border the growing regions; means for scanning a number of said pixels of the list; means for determining, for each said scanned pixel, a value indicative of the similarity of the corresponding motion vector of said scanned pixel and the corresponding set of motion parameters of a growing region that said scanned pixel borders; means for selecting a pixel that has a minimum said value; means for appending said selected pixel to said growing region it borders; means for updating the set of motion parameters of the appended region; means for repeating the operations of the generating means, scanning means, determining means, selecting means, appending means, and updating means until there are no more pixels that border the growing regions; and means for merging neighbouring grown regions to produce a merged region if their similarity is less than a predetermined threshold, wherein said merged regions represent said moving objects and their corresponding sets of motion parameters represent their motion.